Skip to content

fix: key rate-limit hashes in the database and prune expired counters - #19

Merged
operatoruplift merged 3 commits into
feat/store-readinessfrom
fix/rate-limit-retention
Oct 24, 2026
Merged

operatoruplift merged 3 commits into
feat/store-readinessfrom
fix/rate-limit-retention

Conversation

@operatoruplift

Copy link
Copy Markdown
Owner

Merge after #18; this PR is then retargeted to main and rebased. It is stacked on feat/store-readiness because it edits the privacy page that #18 adds, so this diff shows only the rate-limit change.

Findings fixed

Both come from read-only checks on the production Supabase project vhbngdatlowfnwaymvuq:

  1. Counters were never deleted. private.prune_rate_limits() existed, but nothing called it. Production held 6 rows, the oldest from 2026-09-17.
  2. IPs were stored under an unkeyed hash. Each key was stored exactly as the server sent it: scope: plus the first 32 hex characters of SHA-256(ip). Brute force over every IPv4 address recovers the IP.

What changes

All schema changes are in one migration, supabase/migrations/20261008120000_rate_limit_retention_and_keyed_hash.sql.

  • Keyed hash. public.consume_rate_limit now stores HMAC-SHA-256(key, p_key) instead of p_key. The key is 32 random bytes in the new table private.rate_limit_hash_key, seeded by the migration with extensions.gen_random_bytes(32). That table has RLS on, no policy, and no grants to anon, authenticated or service_role, and no function returns the key. The limiter is SECURITY DEFINER with search_path = '', and every object it uses is schema-qualified. No new env var or secret is needed. If the key row ever went missing, the RPC raises 55000; the server already treats any RPC error as "use the per-process limit", so nothing crashes and limiting continues.
  • Retention. A new indexed expires_at column. Every check first deletes expired counters, using pg_try_advisory_xact_lock: if another request is already pruning, it skips rather than waits. Production already has pg_cron (1.6.4), so the migration also schedules buffer-prune-rate-limits every three minutes; it does this only when pg_extension lists pg_cron. That job takes the lock blocking, so it never skips a tick. Windows over 60 seconds are refused; the server uses 60. So no counter outlives the last request by more than 4 minutes. The policy promises 5, which leaves a minute for scheduling jitter.
  • Old rows. The migration deletes every counter stored under the old unkeyed hash. They are one-minute counters, so the only effect is that current windows restart.
  • Privileges. The browser roles get EXECUTE on public functions directly, so revoking it from PUBLIC alone would not be enough. EXECUTE on consume_rate_limit is now revoked from public, anon, authenticated and service_role, and granted back to anon only: the server calls with the publishable key, and the secret gates every call. private.prune_rate_limits() is executable by its owner only, which is how pg_cron runs it. RLS is now on for all three private limiter tables.
  • Version control. The limiter predated the migration history. Its tables are now recorded with their production shape (IF NOT EXISTS, so nothing changes where they exist). A fresh project gets a working limiter plus a generated secret to copy into RATE_LIMIT_SECRET. One side fix: the hosted check p_secret <> (select secret …) let any secret through when the config row was missing (a NULL comparison). The new check requires a matching row.

App. The server code's behaviour does not change. src/server/rate-limit.ts gets updated comments and its first unit tests (14). The tests pin the contract the migration relies on: the RPC receives a fixed-length digest, never the IP, with a 60-second window, and any error, timeout or malformed reply falls back to the per-process limit.

Privacy page wording

These are the only legal changes, in src/app/privacy/page.tsx (added by #18):

Where Before After
Rate limiting, ¶1 "The counter keeps a one-way SHA-256 hash of your IP address, never the address itself, with a request count for the current minute. It holds no account or wallet data." "The counter keeps a keyed hash derived from your IP address (HMAC-SHA-256 under a random key that never leaves our database), never the address itself, with a request count for the current minute. Without that key, the hash cannot be matched to an address by trying every possible one. It holds no account or wallet data."
Rate limiting, new ¶2 none "Each counter is deleted when its one-minute window ends: by the next rate-limit check, or by a cleanup that runs every three minutes. No counter is kept more than five minutes after your last request."
Retention list "Rate-limit counters: a hashed key and a count, with no IP address, account or wallet data." "Rate-limit counters: a keyed hash and a count, with no IP address, account or wallet data. Each is deleted when its one-minute window ends, and no later than five minutes after your last request."

The bolded sentence "IPs are hashed for rate limiting and no address is stored." is unchanged. LEGAL_UPDATED (7 October 2026, shared with the terms) is unchanged; see the owner checklist.

Deploy order: either

The RPC keeps its name, arguments, result shape and secret, and the server keeps sending a digest. So the currently deployed app works against the migrated database, and this branch works against the unmigrated one, exactly as today.

The migration has not been applied to production. I checked hosted prerequisites read-only. The non-superuser postgres role owns the database, the private schema and every limiter table and function, and has USAGE on cron and extensions plus EXECUTE on cron.schedule, extensions.hmac and extensions.gen_random_bytes.

Apply off-peak. Adding the column briefly holds an exclusive lock on private.rate_limits until commit. Requests that wait past the client's 1.5 s timeout fall back to the per-process limit.

Why the key lives in the database

An app-side HMAC keyed with RATE_LIMIT_SECRET would need no table. But it would only take effect once the new app deployed, so the database would keep accepting unkeyed keys from the old deployment. That would force a strict "app first, then migration" order. The secret is also stored in plaintext in private.rate_limit_config, so the trust boundary would be the same. Hashing in the database fixes every caller as soon as the migration runs, in either order.

Verification

  • scripts/verify-rate-limit.mjs (PGlite with pgcrypto, 82 assertions, now in CI) applies the migration as a non-superuser owner, as hosted Supabase does, to two databases:

    • A copy of production before the migration: the exact hosted function and grants, rows from 2026-09-17 and 2026-10-07, and pg_cron metadata.
    • An empty project.

    It covers:

    • old rows deleted and the hosted secret preserved;
    • the stored key equals Node's HMAC-SHA-256(dbKey, p_key) and differs from SHA-256 of the key or the IP, and from the old stored value;
    • each database gets its own key;
    • limits still enforced (allowed ×3, then refused) with scopes kept apart, window rollover renewing the expiry, and rejected calls writing nothing (a window over 60 s is refused);
    • both pruning paths, with the cron job registered once and its stored command run as its user, and safe re-application;
    • a missing key raises 55000;
    • a caller's temp tables and a shadow public.hmac cannot stand in;
    • anon, authenticated and service_role cannot read any limiter table or execute the pruner. Only anon can execute the limiter. RLS still hides the key even after a mistaken grant.
  • Mutation check: deliberately breaking each of these 14 behaviours was caught:

    • storing p_key raw, or storing plain SHA-256;
    • keeping old rows;
    • no prune in the check, or a cron pruner that deletes nothing;
    • no window cap, or no expiry renewal on rollover;
    • revoking only PUBLIC, or leaving the pruner executable;
    • no RLS on the key table;
    • no cron job, or a five-minute cron;
    • a non-empty search_path;
    • restoring the NULL-secret check.
  • scripts/verify-account-deletion.mjs applies every migration to a fresh database, so it now loads pgcrypto through a shared scripts/lib/pglite.mjs, as hosted Supabase has it.

  • Lint, typecheck, unit tests (648), all 5 PGlite verifiers, next build, and the Playwright legal spec (8/8, desktop and mobile, extended to assert the new wording).

  • An independent database review found no critical or high issues. Its medium findings, the cron tick skip, jitter margin, non-superuser apply and window cap, are addressed above.

  • Not covered: concurrency. PGlite has one connection. The locking design means no transaction waits while holding more than one lock that another waits on, but the races themselves are untested.

Owner checklist

  • After feat: dApp Store readiness, keyed rate limits, security-audit fixes and Next.js 16.4.3 #18 merges: retarget this PR to main, rebase, merge.
  • Apply 20261008120000_rate_limit_retention_and_keyed_hash.sql to vhbngdatlowfnwaymvuq, before or after the deploy, off-peak.
  • Afterwards, run these read-only checks:
    • select jobname, schedule, command, active from cron.job where jobname = 'buffer-prune-rate-limits' returns one active row, */3 * * * *.
    • select count(*), min(window_start) from private.rate_limits shows no row older than about five minutes.
    • select length(key) from private.rate_limits limit 1 returns 64.
  • The policy says "we … update the date at the top" when it changes. If feat: dApp Store readiness, keyed rate limits, security-audit fixes and Next.js 16.4.3 #18 is already published to production when this ships, bump LEGAL_UPDATED in src/lib/legal.ts. It is shared with the terms page, so it was left unchanged here.
  • Optional: cron.job_run_details grows by about 480 rows a day from this job, on top of the existing per-minute monitor job. Neither job cleans that table.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JHDgeGLvKoi7gnFqUbJ1sw

operatoruplift and others added 2 commits October 8, 2026 21:04
Read-only checks on production found two privacy defects in the hosted
limiter. Nothing called private.prune_rate_limits(), so counters from
17 September were still stored. Each key was the server's unkeyed
SHA-256 of the client IP, which brute force reverses for IPv4.

20261008120000_rate_limit_retention_and_keyed_hash.sql keeps the
consume_rate_limit signature, result and secret, but stores
HMAC-SHA-256 of the sent key under a random 32-byte key in
private.rate_limit_hash_key (RLS on, no grants to anon, authenticated
or service_role). The limiter is SECURITY DEFINER with an empty
search_path. Every check deletes expired counters, and where pg_cron
is installed (it is on the hosted project) buffer-prune-rate-limits
runs every three minutes. Windows over one minute are refused, so no
counter outlives the last request by more than four minutes. The
migration deletes every counter stored under the old hash, records the
limiter's tables for fresh projects, revokes EXECUTE from browser
roles and service_role, and grants it back to anon only. It also
requires a configured secret: the hosted check passed any secret when
the config row was missing.

The server keeps sending a digest, never an IP, so the migration is
safe before or after the app deploys. rate-limit.ts gets its first
unit tests, pinning that contract and the per-process fallback.
scripts/verify-rate-limit.mjs (82 assertions, in CI) checks the
migration against the hosted pre-migration definitions and an empty
project, applied as a non-superuser owner. The account-deletion
verifier now loads pgcrypto, as Supabase does, because it applies
every migration.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01JHDgeGLvKoi7gnFqUbJ1sw
…olicy

The rate-limiting section said counters keep a one-way SHA-256 hash of
the IP and gave no retention period. It now says they hold an
HMAC-SHA-256 under a random key that never leaves the database, and
that each counter is deleted when its one-minute window ends, at the
next check or by a cleanup every three minutes, so none is kept more
than five minutes after the last request. The retention list says the
same. No other legal text changes, and the browser spec checks the new
wording on desktop and mobile.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01JHDgeGLvKoi7gnFqUbJ1sw
@vercel

vercel Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
buffer Ready Ready Preview Oct 8, 2026 2:18pm UTC

Request Review

The policy promises a new date whenever it changes, and this PR changes how
rate-limit counters are hashed and kept. The terms page shares the date.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_01JHDgeGLvKoi7gnFqUbJ1sw
@operatoruplift
operatoruplift merged commit b0dc0ac into feat/store-readiness Oct 24, 2026
5 checks passed

This branch was successfully deployed

1 active deployment
Preview — 3d29443a Deployed Oct 8, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant